Stochastic Resonance can drive adaptive physiological processes

Stochastic resonance (SR) is a concept from the physics and engineering communities that has applicability to both systems physiology and other living systems. In this paper, it will be argued that stochastic resonance plays a role in driving behavior in neuromechanical systems. The theory of stochastic resonance will be discussed, followed by a series of expected outcomes, and two tests of stochastic resonance in an experimental setting. These tests are exploratory in nature, and provide a means to parameterize systems that couple biological and mechanical components. Finally, the potential role of stochastic resonance in adaptive physiological systems will be discussed. Introduction Stochastic resonance (SR) is a dynamical phenomenon that has been found to exist in both living (Weisenfeld and Moss, 1995) and nonliving (Gitterman, 2009) mechanical systems. Stochastic resonance can be defined as the alteration or enhancement of a signal through the addition of a stochastic, or noisy, component (McNamara and Wiesenfeld, 1989). In more technical terms, Mitaim and Kosko (1998) define stochastic resonance as noise that enhances an external forcing signal in a nonlinear dynamical system. While in some contexts the effect of this addition is masking, in time-dependent oscillatory systems it may also result in nonlinear enhancement (Inchiosa and Bulsara, 1995; Freeman et.al, 1997; Braiman et.al, 1995). In this paper, I will argue that stochastic resonance can be a correlative and perhaps even causal mechanism for adaptive physiological processes in neuromechanical systems. In biological systems, Pitti et.al (2005) has defined resonance due to feedback as something that turns chaotic movements into periodic ones through nonlinear control mechanisms (Gammaitoni, 1995). In general, stochastic resonance can be thought of as a pair of training wheels that guides the dynamics in a particular direction. One instance during which the effects of stochastic resonance are triggered is during environmental switching, in which there is a rapid transition between different environmental contexts. This switching might exist on the scale of circannual (seasonal) or circadian (daily) changes. With regard to an explicit effect on the phenotype, this has been observed in bacteria (Balaban et.al, 2004), mammalian learning and cognitive processes (Monsell, 2003; Lackner and DiZio, 2005) and arguably in metamorphic insects (Nijhout, 1999). The switch can be explicitly defined as a trigger that sets off a host of adaptive processes, which ultimately results in large-scale phenotypic changes. These processes can either be suppressive in that they interfere with previous acquisition of environmental stimuli, or multiplicative in that they enhance the previous acquisition of environmental stimuli. In the parlance of physical systems, the former is an example of masking while the latter is an example of stochastic resonance. In this paper, I will further argue that stochastic resonance is the core mechanism behind a specific process called the adaptive ratchet. An adaptive ratchet can be defined as a directional expression of adaptive capacity, and may be driven by SR in neuromechanical systems. This bears a relationship to facilitated variation (FV) as proposed by Gerhart and Kirschner (2007) and demonstrated in silico by Parter et.al (2008), where systems that transmit genetic variants across generations learn the proper adaptive response. We will return to the predictions of FV as they relate to immediate physiological adaptation. Examples and Predictions To better define the relationship between physiology and performance, two hypotheses were formulated. One involves the role of the stimulus, while the other involves the role of physiological/anatomical characteristics. H1: adding neuromechanical noise (a weighted distortion of movement) will perturb performance in a phasic manner. H2: morphological characteristics of an individual will provide an additional source of stochasticity through dampening and have an effect on optimal performance. Below are examples of the mechanisms and phenomena described in the introduction, and how they are related to the two hypotheses. These include the role of stochastic resonance in adaptive processes, the role of pendular control, and examples from the passive dynamic walking literature. Role of Stochastic Resonance (SR) in adaptive processes. One example of the adaptive ratchet process can be seen in 1-D random walk simulations. The effects of stochastic resonance on a 1D random walk were first demonstrated in silico (Leonard, 1992), and then two years later in a chemical reaction (Leonard and Reichl, 1994). The simplest way to conceptualize a 1-D random walk is to imagine a vehicle moving back and forth on a line segment at random intervals. The null hypothesis predicts two outcomes. One, there is no directional trend to this movement over time. Secondly, the intervals over which the vehicle moves in a single step are randomly distributed. A system that exhibits stochastic resonance should violate both of these outcomes. Stochastic resonance has been found to be an integral feature of adaptive tactile (Chiou-Tan et.al, 1997; Collins et.al, 1996), proprioceptive (Daniel and Tu, 1999), and neuromuscular (Cordo et.al, 1996) functional interaction with features of the movement environment. In an oscillating system, stochastic resonance is expected to exhibit two major features. One of these is a directional tuning, albeit temporary, of the amplitude and/or frequency characteristics. The other attribute is a step distribution that is either Poisson or power law in shape (Matsumoto and Tsuda, 1983). One example of such an oscillating system is a pendulum that is driven by a neuromuscular output. The pendulum provides a sinusoidal signal, while the neuromuscular output provides a patterned signal. When combined in an additive fashion, this physiological output is frequencymodulated. In this way, the output and its feedback to the nervous system are reprogrammed. This provides additional information to the signal that was not a part of the original output. When combined in a synthetic fashion, the forementioned physiological output is both frequencyand amplitude-modulated (for concept, see Haberman, 1998). This additional mode of modulation added even more information to the signal, and is used in providing higherdimensional feedback to the nervous system. Pendular control: a mechanical example of stochastic resonance. One observation from a biologically-coupled pendulum is that the anatomical-mechanical system acts to dampen the effects of interaction with the environment. In a like manner, the effects of this oscillatory system can also be synthetic. When this occurs non-directionally, the amount of time the system is dampened is roughly equal to that during which the effect is synthetic. In biological systems, stochastic resonance is an integral part of the sensorimotor control system (Wolpert et.al, 1995). To show the dynamical properties of the selectively perturbed, oscillatory mechanical stimulus on a biological control system, the pole-balancing task is graphically demonstrated in Figure 1. The pole balancing task is often used to illustrate how vertebrates control movement and balance in a range of environmental contexts (Wolpert et.al, 1995; Tin and Poon, 2005). Figure 1. A simple pendular control model (e.g. pole balancing task). In Figure 1, the base of the pole is moved back and forth orthogonal to the initial state of the pole. Whatever direction the base is moved, inertial forces push the pole in the opposite direction. In order to compensate for this instability, a balancing behavior must respond to move the pole in the opposite direction. This control principle has been demonstrated experimentally for human limb movement (de Guzman, 2004) and in mechanical systems controlled using artificial neural networks (Gomez and Miikkulainen, 1998). Furthermore, this compensatory mechanism must operate both with a short to nonexistent delay and the ability to track subsequent inertial forces in the appropriate direction. Both of these abilities can be couched in terms of closedand openloop control. Examples from Passive Dynamic Walking (PDW). The realm of passive dynamic walker (PDW) robots (McGeer, 1990) also provides evidence of how stochastic resonance might tune key parameters of neuromechanical systems. PDW robots are stripped-down versions of bipedal locomotion, using an inverted pendulum, conservation of motion, and limited artificial intelligence to produce stable movement. In Buchli et.al (2006), a PDW robot was designed with a key parameter called intrinsic gait frequency that governed the extent of oscillation during a single step forward. This parameter was found to self-tune and adapt quickly when other parameters such as body mass were changed. In another case, a PDW was taught to adapt to variations in terrain quickly by 1) limiting the degrees of freedom, and 2) exploiting the stochastic components of the behavior (Tedrake et.al, 2004). In this case, the environmental switch was simulated to produce an adaptive response post-hoc, which is essential in maintaining closed-loop control for neuromechanical stability. The passive dynamic walker model teaches us two lessons regarding the role of stochastic resonance in neuromechanical systems. The first lesson shows that morphological control (particularly morphological dampening) plays a critical role in this effect (Full and Koditschek, 1999; Nishikawa et.al, 2007). In this case, external forcing dampens the amplitude of oscillation in the gait parameter. This leads to a conservation of output energy and greater stability. The second lesson is that more specific: while a neuromuscular system can produce internal forces, external forcing produces external forces that impose selective constraints on the gait parameter. Sometimes this effect occurs at multiple regimes, creating multiple stable states for movements such as walking, running, and shuffling (two refs). Results (Experimental Context) Experimental Tests of SR To further understand the dynamics of SR in a neuromechanical system, a unique experimental design and two statistical tests/comparisons are used. Results from the extended proprioceptive switching experiment is used to demonstrate what occurs when an oscillatory system used to drive movement is perturbed at regular intervals. The results of this experiment are understood by using two tests that uncover patterns associated with stochastic resonance. Test #1: Proportional effect of perturbation. To test the hypothesis that adding neuromechanical noise (a weighted distortion of movement) will perturb performance in a phasic manner, the design of the extended proprioceptive switching experiment included a control condition. This aspect of the experimental design was used to better understand the immediate and cumulative effects of a perturbation on an adaptive system. In Figure 2, the MPO is plotted for each block of each non-control condition. Comparing across conditions, there are three trends. The Late condition shows a gradual adaptation to the naked controller which is temporarily interrupted by a perturbation in learning block 3. The Early condition shows a trend parallel to what is seen in the case of the Late condition, except that this is not in response to a perturbation. In fact, the perturbation only seems to have a strong effect on the learning block. Finally, the interleaved condition exhibits a "V" shaped pattern: the lowest level of MPO for the condition occurs during a non-perturbation treatment flanked by two perturbation treatments. In the case of muscle activity (Figure 3), different trends can be seen. The Late condition shows that TB activity rises gradually before a perturbation and then falls in response to it. The Early condition shows the same trend, but in this case TB activity seems to be suppressed by perturbation and rises/levels-off afterward. In the interleaved condition, TB activity peaks during the same block as MPO activity is minimized. This may suggest that this muscle (located in the humeral segment) plays a role in regulating muscle output given the presence of a perturbation beforehand. In the case of FCR activity, the results are more or less equivocal for the Interleaved and Late conditions, with a limited effect for increasing activity during perturbation in the Early condition. The proportional UMP measurements for TB and FCR (Figure 4) reveal finer detail regarding the effects of perturbation on the neuromechanical system. For UMP-TB, there were three distinct effects. For the interleaved condition, perturbation results in an immediate increase in UMP, and appears to be sustained in the tertiary learning block. The Early condition shows that the removal of a perturbation suppresses UMP in an immediate fashion in that the effect wears off in the next block. The Late condition shows that UMP is also suppressed after a double dose of perturbation is presented. The UMP-FCR measurement shows different effects for the same condition. In the interleaved condition, UMP is stable across all blocks but is consistently lower than the control condition in all cases. For the Early condition, there is a slight decrease in the response to a double dose of perturbation. In addition, there may also be a delayed response to the removal of a perturbation. Finally, the Late condition demonstrates that there is a slight decrease in UMP until the perturbation is introduced. Interestingly, learning blocks 3 and 5 have the same UMP value, which may also be evidence of a transitory effect of introducing a perturbation late in the sequence. Figure 2. MPO by mean across blocks of trials and as a percentage of baseline (in every instance, baseline for block is 100). Test #2: simulation via interpolation across trials and blocks. To test the hypothesis that morphological characteristics of an individual will provide an additional source of stochasticity and have an effect on optimal performance, statistical interpolation and visual inspection were used to evaluate the effects of stochastic resonance over time. To simulate prior probabilities for the conditional probability analysis, a cubic spline is used to build a statistical distribution. This method can also be used to interpolate datapoints across discrete trials and provide a time-series function for each condition (Figures 5 and 6). When plotted out, these splines show how the UMP measurement peaks over the duration of the condition. This is useful in delineating trends in the UMP measurement and the cumulative effects of treatment, as the UMP measurement itself is highly variable across trials. Figure 3. Muscle activity (TB and FCR) by mean across blocks of trials and as a percentage of baseline (in every instance, baseline for block is 100). Figure 4. Unmatched muscle power (UMP for TB and FCR) by mean across blocks of trials and as a percentage of baseline (in every instance, baseline for block is 100). UMP is measured in volts/ft. In the UMP-TB measurement, we can see that there is a spike in the middle of the learning block for the Late condition. We can also see a spike related to the onset of a perturbation for the Interleaved condition. There is also a spike at the end of learning block 2 for the interleaved condition, possibly related to the ongoing perturbation. For the Early condition, there is an elevated UMP measurement over the duration of learning block 3. This may be a side effect of the earlier perturbation. Consistent with this notion of delayed effects, the spline-generated function for the Control condition tends to spike in a number of randomly-chosen locations. In the UMP-FCR measurement, a more variable pattern emerges. For the Late condition, the function spikes quite regularly before the perturbation is introduced (at trial point 46). After the perturbation is introduced, it appears to dampen the UMP indicator until almost trial point 70, where another spike occurs. In the Interleaved condition, there is a large spike during the first perturbation (learning block 2) and another smaller spike during the second one (learning block 4). For the Early condition, the removal of the perturbation seems to dampen the UMP, save for a spike at trial point 72. In the Control condition, there is also a much less pronounced dampening across trials, but since it is not associated with an earlier perturbation, we do not see a pronounced spike near the end of the condition. Discussion and Conclusions A number of conclusions can be raised from this experiment and theoretical prediction. The first of these involves the existence of synchronization between the stimulus (a swinging, weighted device) and alterations in performance over time. In this section, two of these will be highlighted: how a weighted pendulum can trigger a process called facilitated variation, and stochastic resonance being a driver of the adaptive ratchet mechanism proposed here. Figure 5. Unmatched Muscle Power (UMP) for Triceps Brachii (TB). Functions: Interleaved = red, Early = blue, Late = black, control = green. Blocks: Learning 1 = 1-16, Learning 2 = 16-31, Learning 3 = 31-46, Learning 4 = 46-62, Learning 5 = 62-78. Facilitated Variation (FV) and the effects of a coupled pendulum. Using a coupled pendulum or similar device over long time-scales may result in facilitating variation that is not expressed under normal conditions. The two attributes of facilitated variation (FV) as proposed by Kirshner and Gerhart (2007) that are most important to this context are exploratory behavior and reduced pleiotropy. In this case, these mechanisms are active within the same generation, and mimic changes that have already occurred over a number of generations. Exploratory behavior refers to the ability for physiological processes to be temporarily decoupled, thus allowing for adaptation. Reduced pleiotropy is a bit more complex, and involves reducing the number of alternate gene products for a single genetic variant (or allele), thus helping to drive adaptation in a specific direction. Together, these processes derive from and build upon existing variation expressed in the life-history of the current generation (Bergman et.al, 2007). This may lead to individual variations in performance over time and across individuals. Figure 6. Unmatched Muscle Power (UMP) for Triceps Brachii (TB). Functions: Interleaved = red, Early = blue, Late = black, control = green. Blocks: Learning 1 = 1-16, Learning 2 = 16-31, Learning 3 = 31-46, Learning 4 = 46-62, Learning 5 = 62-78. Stochastic resonance as a driver of the adaptive ratchet. The proposed dynamics of stochastic resonance in the context of an adaptive ratchet occurs according to the following steps. First, the variance of frequency and amplitude with regard to oscillations in the coupled system increase but remain randomly distributed. Secondly, the frequency and amplitude characteristics of the system become distributed in a non-random fashion. Thirdly, one end of the distribution becomes selected for, and this range of frequency and amplitude parameters become predominant. This has an effect of creating a new dynamical phase for the system, which can be demonstrated empirically in a time-series representation. Materials and Methods More information about all experimental techniques and methods are described in Alicea (2008). These include the measures used, apparatus, and the basic experimental design. Experimental design for extended prosthetic switching experiment. Thirty-two subjects were used in the extended prosthetic device switching experiment. A 2 (prosthetic device) x 4 (perturbation type) x 5 (learning blocks) x 16 (trials) mixed experimental design was used. The between subjects factors are prosthetic device and perturbation, while the within subjects factors are learning blocks and trials. Prosthetic device setting has two levels: naked controller and physical controller. Perturbation type has four levels: early, alternate, late, and none. Learning type has five levels, learning block 1, learning block 2, learning block 3, learning block 4, and learning block 5. Apparatus protocol for extended prosthetic switching experiment. The simulation portion of the prosthetic device switching experiments will be conducted using the Nintendo® Wii gaming platform (Nintendo Corporation, Kyoto, Japan) using the simulation Wii Sports Golf. The Nintendo® Wii uses several simulation simulation-specific parameters and a motion controller to produce computer-generated simulation action. All simulation-specific parameters, such as wind speed and club surface type, will be held constant. Procedure for tactile surface switching experiment. Surface electrode preparation protocols were followed to acquire recordings from the triceps brachii and flexor carpi radialis. Before beginning, subjects were briefly introduced to the interface. The subject then engaged the active exploration of different sample surfaces over three separate experimental blocks. Each block was preceded by a 30-second baseline acquisition, during which the subject was instructed to freely explore the “no surface” sphere. This surface had no force-feedback associated with it. For each block, subjects were encouraged to explore as much of the surface as they could within a two-minute time window, and received force-feedback by the device as they moved against the surface. Prosthetic manipulation for extended prosthetic switching experiment. The prosthetic manipulation will be achieved by varying the type of input device use for the motion controller. The Wii motion controller uses a gyroscope to translate actions produced by the human into a virtual analogue of physical movement. Mechanically, this is the same control principle that produces inertial feedback and continuous motion on a Segway wheeled transport. While this produces gyroscopic forces of limited scope which act as a source of inertial feedback, the gravity and velocity effects of the simulation physics should be consistent within each condition. The experimental manipulation of the environmental variable will be achieved by the use of a prosthetic device. The purpose of this tool is to simulate a set of environmental conditions that perturb upper-arm morphology and the current physiological state. Physiological activity. To collect electrical potentials from the nervous system and information about internal physiological states, the Biopac MP150 biopotential amplification system will be used (Biopac Systems, Goleta, CA). Two channels will be used to collect these data from the muscles of the forearm and humerus. A template was created in AcqKnowledge® (Biopac Systems, Goleta, CA) to digitize, smooth, and transform the raw data in real time. A sampling rate of 1000Hz was used to collect the raw data. A calculation channel with a window of 10 data points was used to smooth the muscle activity data using a moving average routine, which resulted in a final sampling rate of 100 data points per second. A special calculation channel was used to apply an integrated EMG (iEMG) filter to rectify and mathematically integrate the signal representing muscle activity. The hardware filters were set to a gain of 5000, a high-pass analog filter value of 1Hz, and provided notch filtering of the signal (50dB rejection mode at 50-60Hz). A notch filter was used to remove ambient spectral noise. Muscle activity. Electromyography (EMG – see Appendix A) will be collected using the Biopac MP150 system (Biopac Systems, Goleta, CA) to quantify muscle activity. Multiple segments act semi-independently to determine the consequences of sensory input, feedback, and variability. Two points on the upper arm to record EMG data: the flexor carpi radialis along the forearm and the triceps brachii along the humerus. Muscle activity at these locations can be used to determine arm movements in relation to dynamic upper-body posture. The muscles recorded in these experiments were selected based on two factors: muscles that were located along the dorsal or ventral surface of the arm, and muscles that were involved in the complex reaching movement. Several steps were taken to ensure the internal validity of these measurements. Mapped physiological output measurement. A measurement of mapped physiological output will be calculated for several reaches over the course of a twominute trial. Mapped physiological output will be measured using the equation 







































































MPOt = |Dreq Dmoved| / Dreq [1] where Dreq is the fixed distance a virtual object needs to be moved over a given trial, Dmoved is the distance the virtual object is actually moved resulting from muscle force production captured by the input device, and MPOt is the mapped physiological output for a single reach using the prosthetic device. A single reach will consist of taking a swing with the prosthetic interface. The operator will start each swing aligned with markers on the floor that denote a baseline or resting physical state. That reach will conclude and the next reach will begin once the operator re-aligns with these markers. The distance the ball travels in the simulation and a completion time constant will be collected. For each reach, the simulation will be reset so that the shot is always taken from the beginning of the first hole. Sixteen reaches will be administered across a single experimental block. Unmatched muscle power. When a neuromuscular system interacts with a virtual environment, modification of traditional methods used to quantify muscle output is required. Unmatched muscle power is conceptually similar but not identical to muscle power measurements traditionally used in the literature. Unmatched muscle power will be measured using the equation UMP = RPi / MPOi [2] 
 where UMP is the unmatched muscle power, RP is the raw signal peak over a finite time interval corresponding with the duration of an experimental trial, and MPO is mapped physiological output for the trial corresponding to the RP